Large margin multinomial mixture model for text categorization

In this paper, we present a novel discriminative training method for multinomial mixture models (MMM) in text categorization based on the principle of large margin. Under some approximation and relaxation conditions, large margin estimation (LME) of MMMs can be formulated as linear programming (LP) problems, which can be efficiently and reliably solved by many general optimization tools even for very large models. The text categorization experiments on the standard RCV1 text corpus show that the LME method ofMMMs can largely improve classification accuracy over the traditional training method based on the EM algorithm. Comparing with the EMmethod, the proposed LMEmethod can achieve over 20% relative error reduction on three independent test sets of RCV1.